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DETAILED ACTION 
Summary 

1. Claims 1-6 and 24-36 are pending in the application. Claims 7-23 stand 
as withdrawn from consideration. 

This action is responsive to the election of May 2, 3006. The examiner 
notes that these claims were elected with traverse. 

The applicant argues that the restriction is improper because the search of 
the various inventions claimed are not different and do not represent a search 
burden for the examiner. 

The examiner respectfully disagrees. 

The examiner notes that in the first Office Action, several issues with the 
language of specification were pointed out. It is a matter of record that the 
application claims priority to a foreign invention. Furthermore, the language of 
the claims contained several 35 USC 112 2nd rejections, which were further 
reflective of the unclearly worded specification and claims. The application of the 
art in the first rejection was made in light of the numerous 112 2"^ rejections of 
the claim. 

It is the examiner's position that the clarification of the claims made by the 
numerous amendments to comply with the 112 2"^ requirements resulted in a 
clarification of the inventions claimed by the previously described groups. The 
fact that a prior art search has been done on the first office action is irrelevant, 
given the fact that 3 inventions are contained in the instant application. 

For example, Group II (Claims 7-12, 13 and 14) describes the 
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distinguishing between purchasers and non-purchasers using image recognition. 
This invention is classified in Class 382 subclass 418. Group 1 (elected claims 
1-6, 24-36) provides for correlation between a customer and the customer's 
purchased item information. Determining whether a person in a store is a 
purchaser or a non-purchaser using camera systems is an entirely different 
invention in comparison to correlating a customer with their purchased item using 
image information. The software algorithm used to distinguish between 
purchasers and non-purchasers in a store is solving an entirely different image 
science problem than is one that correlates a customer with their purchased item. 
Therefore the restriction of the invention is proper. 

Although inventions may be classified in the same class, the examiner 
would point out that a search would be made across the entire US patent 
database containing keywords that identify salient features of the different 
inventions. This is done because, for example, Class 345, dealing with displays, 
provides for inventions where the art is analogous because inventors are solving 
similar problems. The examiner would also point out that aside from a keyword 
search in the existing patent database, there are also mandatory non-patent 
literature (NPL) databases within Dialog that are required to be searched. 
Additionally, depending on the claims and the examiner's discretion, there are 
other searches that may be made within ProQuest and also using Google. Since 
the potential areas for searching is so broad, the different inventions as claimed 
are, in fact, a search burden for the examiner. 
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Finally, the examiner would point out that the invention claims priority to 
three (3) different Japanese patent applications. 

Claim Rejections - 35 USC §112 

The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

2. Claims 32-34 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject 
matter which applicant regards as the invention. 

Regarding Claim 32, the limitation cites a camera including a first camera 
and a second camera. Since only one camera is being claimed in the parent 
claim (claim 24), the modification of this limitation into two cameras makes the 
claim indefinite, since it is not clear how one camera in Claim 24 becomes two 
cameras in Claim 32. 

In further regards to Claim 32, the claim describes two separate cameras. 
A first camera obtains an image of a customer who has entered a shop. A 
second camera obtains an image from a customer who has purchased an item. 
The last. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 
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A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for patent or 
(2) a patent granted on an application for patent by another filed in the United States before 
the invention by the applicant for patent, except that an international application filed under 
the treaty defined in section 351 (a) shall have the effects for purposes of this subsection of an 
application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

4. Claims 1, 5, 6, 24, 25, 26, 28, 30 are rejected under 35 U.S.C. 102(e) as 
being anticipated by Pong US 6,237,647 (hereinafter Pong). 

Regarding Claim 24, Pong discloses: 

a camera obtaining image information of a customer in a shop 
to generate an image signal indicating an image of said customer; 

column 2 line 44-46, the system contains a vision system (i.e. with a 
camera that obtains an image) to perform facial identification of the customer (i.e. 
the image signal of the customer's face is generated to indicate an image of that 
customer). 

a first processor receiving said image signal to determine an 
attribute of said customer on the basis of said image; 

column 2 line 44-46, since the system if performing facial recognition, it is 
determining an attribute of the customer (i.e. their identity) on the basis of the 
facial recognition. See Figure 2 #21 & #30. 

a first input unit receiving information of an item purchased in the 
shop by said customer; and 
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column 4 line 13-17, the database receives the information of the item 
purchases (i.e. fuel) purchased in the refueling shop by the customer. 

a first storing section storing said information of said item received 
and said attribute of said customer determined, said information and said 
attribute being correlated \N\th each other. 

Column 2 line 44-46, the vision system confirms the facial identity of the 
system with the transponder information when the person is driving up to refill 
their car. 

Column 4 line 13-15, Once the person has finished refueling, the 
database stores the billing information for the transaction of fuel purchased by 
the customer. Since the system is performing the billing function, it correlates the 
identity of the customer (based on their image) with the amount of fuel they 
purchased. See Figure 2 #28. 

Claims 1, 5 and 6 recite limitations similar to those recited by the rejection 
of Claim 24, and are therefore rejected under the same rationale. 

Regarding Claim 25, Pong discloses: 

wherein said camera is arranged at a predetermined position and 

column 13 line 35-40, the camera is arranged at a predetermined position 
(i.e. at the refueling station) 

obtains image information of said customer at a predetermined 
timing. 
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Column 2 line 40-45, the system uses the facial recognition to confirm the 
individuars identity with the transponder. This occurs prior to the refueling of the 
vehicle (i.e. at a predetermined timing). 

Regarding Claim 26, Pong discloses: 

wherein said position is suitable for obtaining image information of a 
face of said customer. 

Column 2 line 45-46, the camera obtains image information of the face of 
the customer since it is performing facial recognition, thus the position of the 
camera is suitable for obtaining image information from the customer's face. 

Regarding Claim 28, Pong discloses: 

wherein said information of said item includes a name of said item. 

Column 7 line 1-5, information regarding items purchased by the customer 
includes names of several items, not only including fuel but other items 
purchased by the customer at the convenience store associated with the 
refueling station. 

Regarding Claim 30, Pong discloses: 

further comprising a second input unit receiving an attribute of said 
customer, 

column 6 line 40-45, the attribute of the customer from the transponder tag 
is the customer's account number. 
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wherein said first storing section further stores an attribute input via 
a second input unit and different from said attribute determined in said first 
processor. 

Column 7 line 12-13, the customer database can be updated to include 
changes in the customer information - see also column 8 line 40-45, the 
customer can input limited commands through the transponder. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 1 02 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

6. Claim 27 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Pong US 6,237,647 (hereinafter Pong) in view of Kaneko US 6,332,139 
(hereinafter Kaneko). 

Regarding Claim 27, Pong teaches where the vision system above 
identifies a person based on their facial characteristics (i.e. facial recognition). 
Pong teaches that the entire customer interaction can be monitored by the 
system, so that personal reminders (i.e. specific to that individual) can be made. 
For example - col 5 line 9-12, Pong teaches an example where a person can be 
directed to pick up his dry cleaning. 
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However, Pong does not teach where the facial recognition system 
provides for a determination as per: 

wherein said attribute includes at least one of gender and age. 

However, it is old and well known in the art where facial recognition can 
provide for both gender and age, as taught by Kaneko. 

Kaneko teaches that his system can provide for a determination of age 
and gender and that this information (i.e. age and gender) helps to determine a 
product suitable for a customer (column 33 line 25-30). Kaneko teaches that his 
system is suitable for monitoring the inside of a store to understand the situation 
in a store (column 2 line 11-13). 

Pong and Kaneko both address the use of facial recognition systems in 
identifying customers to improve the customer's shopping experience, thus both 
Pong and Kaneko are analogous art. 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the teachings of Pong, where facial recognition is used to 
determine the identity of a customer, to include the teachings of Kaneko, where 
the facial information is used to determine demographic information and related 
product characteristics for that demographic information, because it would 
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automatically provide product recommendations to that customer based on their 
age and gender, and thus improve their shopping experience. 

7. Claim 29 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Pong US 6,237,647 (hereinatter Pong). 

Regarding Claim 29, Pong teaches the use of a facial recognition system 
in conjunction with a refueling system to provide an automated buying 
experience for a customer. Pong does not teach where the elements of his 
system are installed inside of a shop (i.e. a store) as per: 

wherein said camera, said first processor, said first input unit and 
said first storing section are installed inside said shop. 

However, it is old and well known in the art for a buying experience to be 
conducted entirely inside of a store. 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the teachings of Pong, to include where his facial 
recognition and associated elements are installed inside of a store, because it 
would improve the shopping experience by automatically identifying the customer 
so that personal shopping recommendations could be made to that customer. 
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8. Claims 2, 3, 4, 31, 32-36 is rejected under 35 U.S.C. 103(a) as being 
unpatentable over Pong US 6,237,647 (hereinafter Pong) in view of Lu US 
5,331,544 (hereinafter Lu). 

Regarding Claim 2, Pong teaches: 

wherein said recording block further records a second preference of 
said customer, 

column 8 line 43-47, the transponder can record changes in the 
customer's preferences for that visit. 

said second preference being correlated with said data of said item 
obtained, said second preference being determined through a human 
perception. 

column 8 line 43-47, these changes can be input by the customer (i.e. 
determined through that customer's own human perception). The attribute is 
correlated with the customer's purchase in that if the customer is buying fuel, but 
wants a different fuel (e.g. different octane rating), the system correlates the 
change with what the customer wants to buy. 

Pong does not teach where an attribute of a customer is determined by a 
human perception and where that attribute of the customer is correlated with the 
item obtained. 
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Lu teaches where an attribute of a custonner is determined by a human 
perception and where that attribute of the customer is correlated with the item 
obtained, (column 9 line 25-30) The clerk determines that the signature of the 
customer (an attribute of the customer) is determined to be correct, according to 
what is on their store ID card. When the person makes the purchase using this 
check, the purchase is correlated with the person's identity. Note that Lu teaches 
using the store ID card to keep track of frequent shoppers (i.e. to correlate 
shoppers with their purchases) - see column 2 line 10-15). 

Lu teaches where having an attribute based on the human perception of a 
customer's identity can prevent authorized use of a check card to prevent 
financial loss to the store (column 9 line 40-45). 

Both Pong and Lu address the use of computerized facial recognition 
systems to improve the effectiveness of customer identification, thus both Pong 
and Lu are analogous art. 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the teachings of Pong, regarding using facial recognition 
systems to identify customers, to include the step determining a customer 
attribute through a human perception, because it would provide a way to prevent 
financial loss to a store. 
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Regarding Claim 31, Pong teaches a facial recognition and purchasing 
recognition system to provide a system for complete monitoring and automation 
of a customer's buying experience. Pong does not teach: 

further comprising a second processor providing a data analysis 
based on said information of said item recorded and said attribute of said 
customer, 

wherein said camera, said first processor, said first input unit and 
said first storing section are arranged inside said shop as an internal 
device and said second processor is arranged outside said shop and 
capable of communicating with said internal device. 

Lu teaches: 

further comprising a second processor providing a data analysis 
based on said information of said item recorded and said attribute of said 
customer, 

column 4 line 35-40, a central computer (i.e. a second processor) stores 
the market research data from a plurality of stores, -see also column 5 line 9-14, 
the data collected is used for market research, i.e. providing data analysis to 
understand shopper purchasing behavior. 

wherein said camera, said first processor, said first input unit and 
said first storing section are arranged inside said shop as an internal 
device and said second processor is arranged outside said shop and 
capable of communicating with said internal device 



Application/Control Number: 09/904,528 Page 14 

Art Unit: 3623 

column 4 line 56-60, the camera, first processor, input unit and storing 
section are located inside the store -see also column 8 line 16-20. 

Column 5 line 13-14, the central computer for collecting the data is 
connected to the individual stores (i.e. can communicate with them since it is 
collecting the data. 

Lu teaches that his system provides a way to determine shopping 
frequency of customers at a store so that retailers can study shopping versus 
non shopping behaviour (column 1 line 38-42) and that this information is 
valuable for market research studies (column 2 line 11-15). 

Both Pong and Lu address the use of computerized facial recognition 
systems to improve the effectiveness of customer identification, thus both Pong 
and Lu are analogous art. 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the teachings of Pong, regarding using facial recognition 
systems to identify customers, to include the step of collecting the data in a 
central computer from several stores, because it would provide a way to conduct 
market research for a plurality of stores connected to the central computer so as 
to understand shopper purchasing behavior for all these connected stores. 
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Claims 3 and 4 recite limitations similar to those addressed by the 
rejection of Claim 31, and are therefore rejected under the same rationale. 



Regarding Claims 32 and 34, Pong teaches a facial recognition system to 
identify customers who are purchasing items from a store. Pong's system is 
connected to a database so that customer purchases can be billed to the 
customer (i.e. the customer visits that result in sales are counted so that accurate 
billing is made to the customer. Pong further teaches that his system is designed 
to provide complete monitoring and control of the customer shopping experience 
(column 5 line 13-15). 

While Pong teaches obtaining facial information of a customer, as 
discussed above. Pong does not teach two different cameras to distinguish 
between customers who have made and purchase and those who have not 
made a purchase as per: 

Wherein said camera includes a first camera obtaining face 
information of a customer having entered said shop and 

a second camera obtaining image information of a customer having 
purchased an item; 

said first processor receives said image signal generated by said 
first camera and an image signal generated by said second camera, to 
determine an attribute of a customer having an image indicated by an 
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image signal generated by said first camera but not by any image signal 
generated by said second camera; and (as per claims 32 and 34) 

said first storing section further stores information indicating that 
said customer is a non-purchaser, said information being added to said 
determined attribute of said customer (as per claim 32). 

said first storing section further stores information indicating that 
said customer is a non-purchaser, said information being added to said 
determined attribute of said customer (as per claim 34). 

Lu teaches using cameras to distinguish between purchasing and non- 
purchasing customers as per: 

Wherein said camera includes a first camera obtaining face 
information of a customer having entered said shop and 

Column 9 line 54-59, shoppers who have entered the shop and are 
looking at a display have their face captured by a camera. 

a second camera obtaining image information of a customer having 
purchased an item; 

column 4 line 55-60, customers who are purchasing an item (i.e. at the 
checkout lane) have their facial images captured by a camera. 

said first processor receives said image signal generated by said 
first camera and an image signal generated by said second camera, to 
determine an attribute of a customer having an image indicated by an 
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image signal generated by said first camera but not by any image signal 
generated by said second camera; and 

Column 9 line 54-59, customers who look at the display have their facial 
images captured for identification (i.e. their attribute, their ID is determined from 
the image captured by this second camera). 

said first storing section further stores information indicating that 
said customer 

is a non-purchaser (as per Claim 32, and) 

is a purchaser (as per Claim 34), 

said information being added to said determined attribute of said 
customer. 

Column 3 line 35-40, the system is designed to correlate those customers 
looking at the display in the store with those who have made a purchase. The 
shoppers whose faces are imaged at the display are identified (i.e. their attribute, 
their identity is determined). Lu's system stores the information for both 
purchasers and non purchasers in order to determine the effectiveness of the in- 
store display towards customers making a purchase. Lu's system is correlating 
purchasing behavior of both purchasers and non purchases (column 3 line 41- 
44). 

Lu teaches that his system provides a way to determine shopping 
frequency of customers at a store so that retailers can study shopping versus 
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non shopping behaviour (column 1 line 38-42) and that this information is 
valuable for market research studies (column 2 line 11-15). 

Both Pong and Lu address the use of computerized facial recognition 
systems to improve the effectiveness of customer identification, thus both Pong 
and Lu are analogous art. 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the teachings of Pong, regarding using facial recognition 
systems to identify customers, to include the step of using two different camera 
systems to determine which customers are non-purchasers, as per Claim 32, and 
which customers are purchasers, as per Claim 34, because it would provide 
valuable information for retail marketing research studies to improve the 
purchase rate of customers. 

Regarding Claim 33, Pong teaches the facial recognition system as 
discussed above and teaches a first camera, but does not teach: 

wherein said first camera is positioned in a vicinity of an entrance of 
said shop and 

said second camera is positioned in a vicinity of a cash desk of said 

shop. 

Lu teaches a second camera positioned in a vicinity of a cash desk of said 
shop, as discussed above, to capture the facial image of customers who are 
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purchasing items. Lu teaches a second camera that is positioned in vicinity of a 
store display. 

Lu teaches that his system provides a way to determine shopping 
frequency of customers at a store so that retailers can study shopping versus 
non shopping behaviour (column 1 line 38-42) and that this information is 
valuable for market research studies (column 2 line 11-15). 

Both Pong and Lu address the use of computerized facial recognition 
systems to improve the effectiveness of customer identification, thus both Pong 
and Lu are analogous art. 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the teachings of Pong, regarding using facial recognition 
systems to identify customers, to include the step of using two different camera 
systems to determine which customers are non-purchasers, and which 
customers are purchasers because it would provide valuable information for retail 
marketing research studies to improve the purchase rate of customers. 

While Lu does not explicitly teach a first camera in the vicinity of the store 
entrance, it is old and well known in the art of retail for displays to be positioned 
at the entrance of a store. This is done to entice customers to enter the shop. 
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It would have been obvious to one of ordinary skill in the art at the time of 
the invention to further modify the collective teachings of Pong and Lu, regarding 
using two different cameras to conduct market research to determine the 
effectiveness of store displays on purchase behavior, to include the step of 
positioning a camera in the vicinity of a store display in the vicinity of the 
entrance of a store, because it would provide an evaluation of how effective the 
store entrance displays were in enticing customers to make a purchase. 

Regarding Claim 35, Pong teaches using a facial recognition system to 
validate a customer's identity when they are making a purchase. Pong does not 
teach counting a frequency of customer's visit to a store as per: 

a second recording block recording an image signal representative 
of a customer of a shop and a number of said customer's appearance in 
said shop, said image signal representative of said customer and said 
number being correlated with each other; and a third processor 
incrementing by one said number correlated with said image signal 
recorded in said second recording block and found to match said image 
signal generated by said camera. 

Lu teaches counting the frequency of a customer being in a shop as per: 

a second recording block recording an image signal representative 
of a customer of a shop and a number of said customer's appearance in 
said shop, said image signal representative of said customer and said 
number being correlated with each other; and a third processor 
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incrementing by one said number correlated with said image signal 
recorded in said second recording block and found to match said image 
signal generated by said camera. 

Column 2 line 10-12, Lu teaches counting not only the frequency of 
shoppers who appear in a shop but also a temporal distribution of their visits. 

Column 2 line 15-20, The shoppers appearance in a shop is correlated 
with that individual customer so that an analysis can be made of the customer's 
purchasing behavior. See also column lline 38-42, the visits of frequent 
shoppers (i.e. the number of their visits to a store) is used in an analysis of their 
average purchase amount. 

Lu teaches that his system provides a way to determine shopping 
frequency of customers at a store so that retailers can study shopping versus 
non shopping behaviour (column 1 line 38-42) and that this information is 
valuable for market research studies (column 2 line 11-15). 

Both Pong and Lu address the use of computerized facial recognition 
systems to improve the effectiveness of customer identification, thus both Pong 
and Lu are analogous art. 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the teachings of Pong, regarding using facial recognition 
systems to identify customers, to include the step counting the frequency of 
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shoppers visiting the store because it would provide valuable information for 
retail marketing research studies to understand the shopping behavior of 
customers. 

Regarding Claim 36, Pong teaches a system that uses a facial recognition 
system to identify existing customers. Pong does not teach adding a previously 
unrecognized customer as per: 

wherein said third processor additionally records in said second 
recording block said image signal of said customer generated by said 
camera if said image signal of said customer generated by said camera is 
not present in said second recording block. 

Lu teaches recording images of customers that have not been previously 
recorded or recognized as per: 

wherein said third processor additionally records in said second 
recording block said image signal of said customer generated by said 
camera if said image signal of said customer generated by said camera is 
not present in said second recording block. 

Column 8 line 32-36, customers images may be used to populate a 
database if those customers appear in the store so that their images are 
captured by the camera. These images are used to populate a database in order 
to determine the frequency of shoppers' appearances and to correlate this with 
their purchases. 
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Lu teaches that his system provides a way to determine shopping 
frequency of customers at a store so that retailers can study shopping versus 
non shopping behaviour (column 1 line 38-42) and that this information is 
valuable for market research studies (column 2 line 11-15). 

Both Pong and Lu address the use of computerized facial recognition 
systems to improve the effectiveness of customer identification, thus both Pong 
and Lu are analogous art. 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the teachings of Pong, regarding using facial recognition 
systems to identify customers, to include the step of populating a database of 
customer images to track the appearance and buying behavior of these 
customers because it would provide valuable information for retail marketing 
research studies to understand the shopping behavior of customers. 

Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

US 6990217 Moghaddam discloses a gender classification with vector 
machines. 

JP 2000251077 A by Goon discloses a customer face recognition method. 
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US 6386323 by Ramachandran discloses a cash dispensing method and 
system. 

US 6522772 by Morrison discloses a self-checkout terminal using a 
biometric sensing device. 

1 0. Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Jonathan G. Sterrett whose telephone 
number is 571-272-6881 . The examiner can normally be reached on 8-6. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Tariq Hafiz can be reached on 571-272-6729. The fax 
phone number for the organization where this application or proceeding is 
assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). 
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